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ABSTRACT 

Confirmatory Factor Analysis (CFA; LISREL) was used 
to: (1) validate the Self Description Questionnaire III (SDQ ill) 
subscales measuring general, school, English, and mathematics 
self-concepts for a sample of 696 (265 low track and 613 high track) 
students in grades 11 and 12, and (2) test the equivalency of the 
factor structure across academic tracks* The results confirm a 
four'factor structure and reveal all factor loadings and covariances 
to be invariant across groups; 11 of 21 uniquenesses were 
non-invariant* Although, in a strict statistical sense, the 
differentially reliable items bore implications of bias in favor of 
the high track, these discrepancies, on the basis o£ absolute values, 
were judged to be of little practical significance* Overall, in light 
of the stringency of LISREL CFA procedures in general, and those used 
in this study in particular, the SDQ III demonstrated exceptionally 
superior psychometric properties* (Author) 
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Abstract 

Confirmatory factor analysis (CFA; LISREL) was used to (a) 
validate the Self Description Questionnaire III (SDQ III) 
subscale measuring general, school, English, and mathematics 
self-concepts for a sample of 898 (285 low track, 613 high 
track) grades 11 and 12 students, and (b) test the equivalency 
of the factor structure across academic track. The results 
confirmed a 4-factor structure and revealed all factor loadings 
and covaiiances to be invariant across groups; 11 of 21 
uniquenesses were nonin variant * Although, in a strict 
statistical sense , the differentially rel iable items bore 
implications of bias in favor of the high track, these 
discrepancies, on the basis of absolute values, were judged to 
be of little practical significance* Overall, in light of the 
stringency of LISREL CFA procedures in general, and those used 
in this study in particular, the SDQ III demonstrated 
exceptionally superior psychometric properties. 
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Testing for Equivalent Factorial Validity Across Academic 
Track: A Confirmatory Factor Analysis of the Self Description 

Questionnaire III 

The Self Description Questionnaire III (SDQ III; Marsh & 
O'Neillt 1984) is designed to measure multidimensional academic 
and nonacademic SCs for late adolescents^ and is theoretically 
linked to the hierarchical model of self-concept (SC; 
Shavelson, Hubner, & Stanton, 1976), Development of the SDQ III 
was based on construct validation research bearing on two 

earlier versions of the instrument the SDQ for 

preadolescentSt and the SDQ II for early adolescents (see 
Marsht BarneSt & Hocevar. 1985 for a research summary) . 

The SDQ III contains 136 items measuring 13 SC facets — 
one general SC^ three academic SCs (English^ mathematics » 
general school), and nine nonacademic SCs (physical ability* 
physical appearance* social (same sex)* social (opposite sex)* 
parent relations* emotional stability, problem solving/ creative 
thinking* religion/ spirituality , and honesty/ reliability) * Only 
the general and academic SC subscales are relevant to the 
present investigation. 

Exploratory and confirmatory factor analyses of the SDQ III 
have yielded we 11 -defined general* and academic SC facets that 
were relatively distinct from each other (Byrne & Shavelson, 
1986; Marsh & O'Neill* 1984; Marsh et al.* 1985; Marsh* 
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Richards, & BarneSt in press). Internal consistency reli^ibility 
coefficients ranging from 0,79 to 0,95 (mean q = 0.90; Byrne & 
Shavelson, 1986; Marsh & O'Neill, 1984; Marsh et al,, 1985), 
and test-retest reliability coefficients ranging from 0,66 to 
0,94 (mean £ = 0,86; Marsh et al,, in press) have been 
reported. Tests of factorial invariance across gender have 
demonstrated equivalence for all but two item-pairs of the 
General Self subscale (Byrne, in press a). Finally^ results 
from multltrait-multimethod analyses have shown strong evidence 
of convergent and discrimiijant validities (for a review, see 
Byrnet in press a) , 

Taken together » these findings provide strong support for 
the SDQ III as a potentially reliable and valid measure of 
adolescent SC, However, the assuiaption of equivalent factorial 
validity across levels of intellectual ability has not beon 
directly tested. Indeed » previous research ^^^s shown the 
emergence of differential factor structures based on responses 
by children of different ability levels ( see e , g , , Byrne & 
Schneider, 1988; Silon & Harter, 1985), Thus, the validity of 
findings from research and program evaluations bearing on 
ability-group comparisons is dependent upon the factorial 
Invariance of the measuring instrument. The purposes of the 
present study were twofold; (a) to test for the factorial 
validity of the SDQ III sub scales measuring general, school, 
English, and mathematics SCs for low- and high-track high 
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school students, and (b) to test for the factorial invariance 
of these subscales across academic track. 

Method 

Samp le and Procedure 

The present data were derived from a larger study that was 
designed to validate the structure of adolescen*^ SC (Byrne & 
Shavelson, 1986) t Following xistwise deletion of missing data, 
the present sample comprised 285 low-track, and 613 high-track 
students from two high schools in Ottawa* Canada* The data were 
slightly negatively skewed with values ranging from -1*60 to 
0*61 Cf = -0*43) for the low track, and froit -2*35 to 0*07 (I = 
-0*79) for the high track; kurtosis ranged from -*82 to 2*76 (X^ 
= 0*19) for the loV track, and from -1*06 to 5*23 (£ = 0*50) 
for the high track* Given mean skewness ranges between -1,00 
and +1*00, however* lictle distortion to the parameter esti- 
mates was expected (see Muthen & Kaplan, 1985)* (For a more 
extensive description of academic tracks, sampling procedures, 
and instrument administration, see Byrnet in press b)* 

Instrumentation 

The SDQ III is structured on an B-point likert-type scale 
with responses ranging from "1-Def initely False" to "B- 
Definitely True*** The General-Self subscale contains twelve 
items and was used to measure general SC* The Academic SC, 
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Verbal SC t and Mathematics SC subscales each contain ten items 
and were used to measure general school. English^ ^nd 
mathematics SCs^ respectively. 
AnalTsi-g of the Data 

All responses to negatively worded items were reflected so 
that the highest response code indicated a positive rating of 
SC. Using confirmatory factor analytic procedures (LISKEL VI; 
Joreskog & Sorbom, 1985), the data were analyzed in two stages. 
First, the factorial validity of the SDQ III was tested 
separately for low- and high- track students;. Second » the 
factorial invariance of the SDQ III was tested across academic 
track , 

Consistent with Marsh and associates* research on the SDQ 

Illt all analyses were conducted on item responses formed in 

pairs (for a description and rationale for this procedure* see 

Marsh & 0*Neillt 1984; Marsh et al,, in press). Assessments of 

model fit were based on the following criteria: (a) the chi 

square ( ) likelihood ratio test^ (b) the X^^^^^S^^ees of 

freedom ratio* (c) Bentler and Bonett's normed index of fit 
1 

(BBI). and (d) T-values* normalized residuals and modification 
indices, all provided by the LISREI program. 

Although EFA is widely used in construct validation 
research, it is limited in its ability to: (a) yield unique 
factorial solutions, (b) define a testable model * (c) assess 
the extent to which an hypothesized model fits the data, and 
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suggest alternative parameterization for model improvement and, 
(d) adequately test factorial invar iance across groups 
(Fornell, 1983; Long, J983; Marsh & Hocevar, 1985), CFA, on the 
other hand, yields this Information and is therefore a more 
powerful test of factorial validity. The CFA model in the 
present study hypothesized a priori that: (a) responses to the 
SDQ III could be explained by four factors, (b) each item-pair 
would have a n on- zero loading on the SC factor it was designed 
to measure, and zero loadings on all other factors, (c) the 
four factors would be correlated and, (d) the error /uniqueness 
ternts for the it em -pair variables would be uncorrelated. 
Finally, &ince the same it ems were administered to both low- 
and high- track students, the measurements were hypothesized to 
be invariant. This hypothesis was tested directly by 
constraining the factor loadings and uniquenesses to be 
equivalent across track 

Results and Discussion 
Confirmatory Factor Analyses 

Although, for both tracks, the hypothesized 4-factor model 
represented a statistically unacceptable fit to the data (low 
track, ^^j33 425,18; high track, X^^^^ 805,45), the normed 
fit index for the high track represented a psychometrically 
reasonable fit to the data (BBI = ,91) indicating that over 90% 
of the data covariation was accounted for; the index of fit for 
the low track was less adequate (BBI = ,86), The factor loading 
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and uniqueness estimates for these models are presented in 
Tables 1 and 2, 



Insert Tables 1 and 2 about here 



One factor considered to be instrumental in undermining the 

2 

model fit was the presence of corrlelated uniquenesses. Indeed, 
previous research has demonstrated that LISREL models involving 
psychological constructs in general (see e.g., Joreskog , 1932 ; 
Newcomb, Huba, & Bentler, 1986; Huba, Wingard, & Bentler, 
1981), and the SC construct in particular (see e-g-, Byrne & 
Shavelsont 1986, 1987) , often requires the researcher to 
specify correlated uniquenesses in order to obtain a well- 
fitting model; such parameter specifications, of course, being 
theoretically and empirically jusified (see Fornell, 1983; 
Gerbing & Anderson, 1984) , Correlated uniquenesses frequently 
result from non random error introduced by a particular 
measurement method; one example is that of method effects due 
to the item format associated with subscales of the same 
measuring instrument. 

To investigate the misfit in the model, then, a sensitivity 
analysis was conductei (see Byrne, Shavelson^ & Muthen^ 1987; 
Tanaka & Huba^ 1984). As sucht model fitting for each track was 
continued beyond the initially fitted models. Several 
additional modifications that included both correlated 
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uniquenesses and secondary loadings (itei.-pair loadings on 
no n- tar get factors) , resulted in a statistically better fitting 
model for both the low track Cx^ = 192.48, p = .06; BBI = 

.94) and the high track 143 - 171.52, p = .10; BBI - .98). 
Given the probability of method effects as noted earlier, along 
with the known looderate correlations among the four SC factors 
under study, these parameters were not unexpected* 

Several considerations » however , bore on the decision to 
reject these final models in favor of the more parsimonious 
intial models* First, the uniqueness co variance estimates, 
while statistically significant, were relatively minor* ranging 
from -*14 to *15 (X, ^ *05) for the low track, and from "*06 to 
*27 (1 = *04) for the high track* Second, the estimated 
secondary factor loadings, while statistically significant, 
were also relatively minor, ranging from -*30 to (X^ = *04) 

for the low track, and from -*24 to .29 (X = *03) for the High 
track * Third , the estimated factor loading and factor variance- 
covariance estimates in the final model correlated *93 and *99 
respectively, for the low track* and *94 and *97 respectively, 
for the high track, with those in the Initially hypothesized 
model (see Byrne et al*, i987; Newcomb et al*, 1986; Tanaka & 
Huba, 1984) ; these results substantiated the stability of the 
initial models* Fourth, although each of the model respeci- 
fi cations resulted in a statistically significant improvement 
in model fit, these increments, based on the normed index of 
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fit, were considered of little practical significance (see 
also, Huba et al,, 1981), Fifth, the sensitivity of the 
likelihood ratio test to trivial departures ol the observed 
from an hypothesized modelt with large samples, is now widely 
known (see Bentler & Bonett, 1980; Huba et al,, 1981; Marsh & 
Hocevar, 1985), Finally, given the exploratory nature of these 
supplementary analyses and thus, the risk of capitalization on 
chance factors (see Long, 1983) , the estimates derived from 
tiiv'se f inal models were considered dubious. For these reasons* 
then, the initial model for each track was used as the baseline 
model in tests of in variance. 
Factorial In variance Across Academic Track 

The next step in the analyses involved estimating 
parameters simul taneously for low and high tracks, in order to 
test for equivalencies of item-pair measurements, and factor 
covariances. These results are summarized in Table 3, 



Insert Table 3 about here 



Tests of invariance involved specifying a model in which 
certain factor loading parameters were constrained to be equal 
across track and then comparing that model with a less 
restrictive model in which these parameters were free to take 
on any value. Since the difference in (^X^Hs distributed as 
X , with degrees of freedom equal to the corresponding 
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difference in degrees of freedom, it provides a basis for 
determining the tenability of the hypothesized equality 
constraints; a sisnificant indicating noninvariance. For 

example, Model 2 in which all factor loadings were specified as 
equal across track was compared against Model 1 in which only 
the number of factors was held invariant; the pattern of factor 
loadings was unconstrained* The difference in was 23,91, 

17 

which was not significant. This finding indicated that the 
pattern of factor loadings was equivalent across academic 
track; items were thus measuring designated SC facets in the 
same way for each track. 

Some researcher s have argued that claims of factorial 
invar iance should provide additional evidence of equivalent 
uniquenesses across groups (see e,g,, Ben son, 1987), Non- 
equivalent uniquenesses , then , -^ould suggest that an instrument 
is more reliable for one group than it is for another. Green 
(1975) has further contended that in such instances, the 
instrument is clearly measuring something different fci each 
group and, therefore, is differentially ^alid* 

To test for the equivalence of uniquenesses across track, a 
model in which the number of factors, pattern of factor 
loadings and all uniquenesses were constrained to be equal 
across track (Model 3) was compared with one in which the 
uniquenesses were unconstrained (Model 2); the X differential 
was highly significant (^X^ = 139, 13), Tests of in variance 
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proceeded ne*^ t to identify it em-- pa ir measuremen t s that were 
differentially reliable across trick* As such» a series of 
mo dels was specified in which all factor loadings* and one 
uniqueness under test were held invariant across track* Given 
findings of invariant uniquenesses^ however ^ hese parameters 
too were cumulatively constrained to be equal across track* For 
example^ in testing for the invariance of MSC5 (item-pa^'r 21) , 
all factor loadings and 9 uniquenesses, in addition to the one 
being testedj were held in va riant ( see nonasterisked unique- 
nesses in Tables 1 & 2)* Finally, a test for the invariance of 
factor covariances resulted in a Ax^ that was not significant 
(AX%^ = 41*41)*' 

Concl usions 

CFA procedures were used to test the factorial validity of 
the general and academic subscales of the SDQ III* The results 
demonstrated a well-defined factor structure yielding one 
general SC facet, one school SC facet, and two subject matter 
facets — English SC and mathematics SC* Tests of invariance 
revealed it era- pair measurements that had equivalent factor 
loadings f albeit partially nonequivalent uniquenesses; all 
ft^ctor covariances were equivalent* 

Th^ major finding was that while the SDQ III measured the 
same SC facet in the same scale units for both tracks (i*e*, 
invariant factor loadings), it did so with a differsntial 
degree of reliability (i*e. , noninvariant uniquenesses)* 
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Specifically, 10 items were more reliable for the high, than 
for the low track; the reverse was true for one item only. 
Since reliability casts the upper bound for validity, this 
finding translates into one of differential validity which, in 
^urn, implies item bias in favor of the high track for 10 of 
the 11 items (see Benson, 1987; Green, 1975) . 

However, in assessing the factorial in variance of a 
measuring instrument , practical, as well as statistical 
siEtnificance, must be taken into account. The practical 
importance of differences in item reliabilities can be more 
realistically judged by examining the absolute magnitude of 
these discrepancies (see Werts, Rock, Linn, & Joreskog, 1976)* 
As such. The discrepancies were considered of little practical 
significance* Indeed, it is highly likely that sample-speci f ic 
artifacts in the data, such as differential correlated errors 
across track, contributed importantly to the non-equivalence of 
item-pair reliabilities (see e*g*. Marsh, in press; Tanaka & 
Huba, 1984)* Thus, while the claim that factorial validity is 
justified only with evidence of both invariant factor loadings 
and uniquenesses is technically correct (see Benson, 1987), 
this criterion is considered to be excessively stringent 
(Muthen , personal communication , January, 1987)* For all 
practical purposes, then, mean comparisons across academic 
track, should not be detrimentally afiected by these 
differential item-pair reliabilities in the SDQ III* 
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Overall, given the stringency of the LISREL CFA procedures 
in general, and the degree of statistical rigor applied in thi3 
study in particular, the SDQ III demonstrated excellent 
psychometric properties. The instrument is also easily 
administered, easily scored, time-efficient , and easily adapted 
to specific assessment needs (i*e.f each subscale is an 

independently valid measure of one particular SC facet) all 

Important consideration*? for testing at the secondary school 
level. Given the scarcity of psychometrically-sound measures of 
adolescent SC, coupled with the concerns of school counselors, 
school psychologists, and school administrators for the 
self- perceptions of low ability students in academically- 
tracked schools, the SDQ III can become an invaluable 
assessment tool for these professionals in their measurement of 
general, school, English, and mathematics SCs for high school 
s t uden t s . 
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Footnotes 

A X /df ratio < 1-50 (Muthen , personal communication , 
January, 1987), and a BBI >,90 (Bentler & Bonett, 1980) 
represent reasonable model approximations to the observed 
data , 

The term "uniqueness" is used in the factor analytic 
sense to mean a composite of specific and random measurement 
error which, in cross-sectional studies, cannot be 
separated (for an extended discussion, see Gerbing & 
Anderson , 1984) , 

Equality constraints were specified for all factor loadings, 
all covariances, and all invariant uniquenesses (see Tables 
1 and 2), 
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Table 1 

Item Factor loading. Uniqueness, and Reliability Estimates for 
low Track (n » 285) ^ 



Factors 









Pnni i^h 




1 In IriMAn AC c 
Ull ll^vicilco «> 










sr 




l\6 t 1 aU 1 t 1 LJ 


GSC 1 


1 000^ 


0 0 


n n 


0 0 


289* 


711 


GSC 2 


.9ii7 


M * u 


n n 

u * u 


0 0 


349 


* V ^ 1 


GSC 3 


.974 


0.0 


0.0 


0.0 


.326 


.675 


GSC 4 


.749 


0.0 


0.0 


0.0 


.601** 


.561 


GSC 5 


.927 


0.0 


b.O 


0.0 


.390** 


.611 


GSC 6 


.770 


0.0 


0.0 


0.0 


.578 


.422 


ASC 1 


0.0 


1.000^ 


0.0 


0.0 


.501 


.499 


ASC 2 


0.0 


1.159 


0.0 


0.0 


.330** 


.670 


ASC 3 


0.0 


1.220 


0.0 


0.0 


.257*** 


.744 


ASC 4 


0.0 


1.108 


0.0 


0.0 


.387*** 


.614 


ASC 5 


0.0 


.984 


0.0 


0.0 


.517** 


.483 


ESC 1 


0.0 


0.0 


1.000® 


0.0 


.678 


.322 


ESC 2 


0.0 


0.0 


1.058 


0.0 


.640 


.360 


ESC 3 


0.0 


0.0 


1.040 


0.0 


.652 


.348 


esc 4 


0.0 


0.0 


1.079 


0.0 


.625* 


.375 


ESC 5 


0.0 


0.0 


.791 


0.0 


.799** 


.202 


MSC 1 


0.0 


0.0 


0.0 


1.000® 


.572** 


.428 


MSC 2 


0.0 


0.0 


0.0 


1.244 


.337 


.663 


MSC 3 


0.0 


0.0 


0.0 


1.301 


.275** 


.725 


MSC 4 


0.0 


0.0 


0.0 


1.266 


.314 


.686 


MSC 5 


0.0 


0.0 


0.0 


1.032 


.499 


.501 


* p < .05 


** p < .01 


**«p 


< .001 









* Unstandardlzed solution 

" All factor loadings were statistically significant and Invariant across track 
^ All uniquenesses were statistically significant 
° Asterisked values Indicate nonlnvarlance 
^ Fixed parameter 



SC = self-concept; GSC « general SC; ASC » school SC; ESC = English SC; 

MSC ' mathematics SC; GSCl = items 1 and 2 measuring GSC; GSC2 = items 3 and 4 

measuring GSC» MSC5 = items 9 and 10 measuring MSC 
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Self Description Questionnaire III 

Table 2 

Item Factor load1ng« Uniqueness, and Reliability Estimates for 
High Track (jL' 613) ^ 



Factors 



1 wm pal r 


^ Ana & 1 




Ul 9 1 1 3 II 


M & ^ Kama ^ 4 c 

na Lnsina Lies 


t tn 4 1 lan ac c 
Uni^U9n955 




-API 1 W AH^& rt ^ p 








sc 


Kc 11 aOI 1 1 


U^U i 


1 f\f\f\ ^ 
i t uuu 


Ut u 


u.u 


Ut U 




. / 3 J 






Ut u 


u.u 


U.U 




.0 J/ 


GSC 3 


1.004 


0.0 


0.0 


0.0 


.240 


.759 


GSC 4 


873 


n 0 

U * V 






426** 


574 


GSC 5 


1.009 


0.0 


0.0 


0.0 


.232** 


.767 


GSC 6 


.851 


0.0 


0.0 


0.0 


.455 


.545 


ASC 1 


0.0 


1.000® 


0.0 


0.0 


.587 


.413 


ASC 2 


0.0 


1.328 


0.0 


0.0 


.271** 


.728 


ASC 3 


0.0 


1.400 


0.0 


0.0 


. 190*** 


.809 


ASC 4 


0.0 


1.316 


0.0 


0.0 


.284*** 


.715 


ASC 5 


0.0 


1.214 


0.0 

1.000* 


0.0 


.391** 


.609 


ESC 1 


0.0 


0.0 


0.0 


.616 


.384 


ESC 2 


0.0 


0.0 


1.094 


0.0 


.541 


.460 


ESC 3 


0.0 


0.0 


1.051 


0.0 


.576 


.424 


ESC 4 


0.0 


0.0 


1.116 


0.0 


.522* 


.478 


ESC 5 


0.0 


0.0 


.619 


0.0 „ 
1.000* 


.353** 


.147 


MSC 1 


0.0 


0.0 


0.0 


.347** 


..653 


MSC 2 


0.0 


0.0 


0.0 


1.087 


.228 


.772 


MSC 3 


0.0 


0.0 


0.0 


1.134 


.161** 


.840 


MSC 4 


0.0 


0.0 


0.0 


1.103 


.207 


.794 


MSC 5 


0.0 


0.0 


0.0 


1.006 


.340 


.661 


* p < .05 


** p < .01 


***p 


< .001 









Unstandardlzed solution 
^ All factor loadings were statistically significant and Invariant across track 
^ All uniquenesses were statistically significant 
^ Aster-* ''ed values Indicate nonlnvarlance 
* Fixed Parameter 



SC = self-concept; GSC » general SC; ASC = school SC; ESC = English SC; 

MSC = mathematics SC; GSCl » Items 1 and 2 measuring GSC; GSC2 « Items 3 and 4 

measuring GSC» MSC5 * Items 9 and 10 measuring MSC 
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Self Description Questionnaire III 

Table 3 

Simultaneous Tests of Invarlance for Item-pair Measurements 

Competing Models ,^2 ^^2 y,2f^f ggj 

0 Null Model 11,895.61 420 28.32 

1 Four SC factors 1,230.64 366 — 3.36 .90 
Invariant 

2 Model 1 with all 1,254.55 383 23.91 17 3.28 .89 
factor loadings 

Invariant 

3 Model 2 with all 1,393.68 404 139.13 21*** 3.45 .88 
uniquenesses 

Invariant* 

4 Model 2 with 1,275.83 399 21.28 16 3.20 .89 
Invariant 

uniquenesses and 
factor covarlances 
Invariant 



* Results from tests for the Invarlance of uniquenesses across track are 
reported In Tables 1 and 2. 

SC » self-concept 



ERIC 



